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Scientific  Work  done  during  reporting  period 

1 .  Abstract 

This  document  forms  the  final  report  on  the  development  of  a  stereo  vision  system  that  can  be  mounted 
inside  a  rolling  tire,  known  as  T2-CAM  for  Tire-Terrain  CAMera.  The  T2-CAM  system  measures  the  three 
dimensional  deformation  of  the  inside  of  a  tire  as  it  rolls  over  the  terrain.  The  system  is  designed  to  work  in 
conjunction  with  the  previously  developed  Vehicle  Dynamics  Group  (VDG)  wheel  force  transducer  that  can 
measure  the  forces  and  moments  between  a  tire  and  terrain  under  off-road  conditions.  The  combination  of 
these  two  systems  provide  the  three  dimensional  geometry,  describing  the  tire-terrain  interface  from  inside 
the  tire,  in  combination  with  all  forces  and  moments  acting  on  the  tire.  The  report  describes  the  experimental 
results  obtained  with  the  T2-CAM  System.  Results  indicate  that  the  system  is  feasible  and  can  provide 
valuable  data  to  researchers  working  in  the  areas  of  terramechanics  as  well  as  tire  modeling  and  testing.  The 
system  meets  the  initial  specifications  and  is  expected  to  provide  much  needed  new  insight  in  understanding 
of  tire-terrain  interaction. 

2.  Problem  statement 

Obtaining  tire  characteristics  on  off-road  terrain  for  use  in  tire  models,  as  well  as  suitable  tire  models 
represent  a  significant  research  challenge.  A  first,  but  extremely  important  step  in  this  research  is  to  develop 
suitable  tire  test  equipment.  Due  to  the  difficulty  of  simulating  off-road  terrain  under  laboratory  conditions, 
field  test  equipment  is  required  that  can  determine  tire  characteristics  on  vehicles  whilst  driving  over  these 
terrains.  The  primary  research  problem,  addressed  by  this  proposal,  is  the  development  of  field  test 
equipment  that  can  measure  dynamic  tire  deformation  from  inside  a  tire  rolling  over  rough  and/or  soft  terrain. 
The  tire  deformation  can  then  be  compared  to  the  measured  forces  and  moments  with  the  use  of  the  Vehicle 
Dynamics  Group  (VDG)  wheel  force  transducer  as  described  in  the  Wheel  Force  Transducer  Research  and 
Development  final  report  [1]. 

3.  Objectives 

In  order  to  obtain  tire  characteristics  over  rough  off-road  terrain,  cost-effective  field  test  equipment  is  required. 
The  proposal  therefore  has  three  main  objectives  [2,3]: 

a)  Develop  a  stereo  vision  system  for  mounting  inside  a  tire  to  measure  tire  deformation  on  a  vehicle 
whilst  driving 

b)  Develop  software  to  post  process  the  measured  data 

c)  Manufacture,  validate  and  calibrate  the  stereo  vision  system  inside  the  tire. 

The  first  objective,  namely  to  develop  a  stereo  vision  system  for  mounting  inside  a  tire  to  measure  tire 
deformation  on  a  vehicle  whilst  driving,  was  achieved  and  described  in  [4]. 

The  second  interim  report  [5]  described  the  development  of  the  software  used  to  post  process  the  measured 
data  under  point  b)  above  as  well  the  manufacturing  of  the  hardware  (point  c  above).  The  validation  and 
calibration  of  the  complete  stereo  vision  system  inside  the  tire  under  point  c)  is  covered  in  this  final  report. 

4.  Stereo  vision 

Stereo  vision  works  on  the  same  principles  as  human  vision.  In  very  basic  terms,  two  cameras,  representing 
the  human  eyes,  are  used  to  observe  the  same  object  from  a  slightly  different  viewpoint.  The  distance 
between  the  object  and  the  camera  determines  where  the  object  is  projected  onto  the  image.  Objects  which 
are  further  away  from  the  cameras  show  less  horizontal  displacement  difference  between  images,  termed  the 
disparity  of  the  object.  By  finding  corresponding  pixels  between  the  images  a  map  disparity  map  can  be 
generated.  Re-projection  is  then  used  to  determine  the  three  dimensional  (3-D)  position  of  an  object  from  the 
two  images.  Figure  1  shows  the  different  images  seen  by  each  camera  and  how  depth  affects  the  re¬ 
projected  objects  and  their  disparity. 
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Figure  1  -  How  stereo  vision  works 


5.  Post  processing  software  development 

The  calibration  process  (shown  in  Figure  2)  was  implemented  in  a  piece  of  software  developed  as  part  of  this 
project  and  aimed  to  make  the  calibration,  visualization  and  stereo  camera  setup  easier.  The  software  was 
built  for  the  Windows  operating  system  using  the  Open  Computer  Vision  (OpenCV)  libraries  [6]  for  computer 
vision  and  the  Qt  library  [7]  for  the  user  interface. 

The  software  has  the  following  features: 

•  Generates  calibration  data  for  stereo  cameras  using  any  asymmetric  dot  calibration  surface  and 
saves  the  data  to  an  “.xml”  file. 

•  Rectifies  and  show  images  by  applying  calibration  data  (from  file  or  newly  generated)  to  input  data. 

•  Allows  input  from  video  files  or  cameras  connected  to  the  computer  via  USB. 

•  Provides  full  control  of  video  playback  with  options  for  pause,  fast  forward,  rewind,  step  forward  by  a 
frame  or  step  back  by  a  frame. 

•  Displays  disparity  between  the  rectified  images  using  one  of  two  matching  algorithms  (Block 
matching  or  Semi  Global  Block  matching). 

•  Tunes  stereo  matching  parameters  to  improve  disparity  quality. 

•  Applies  a  color-map  onto  the  disparity  to  allow  clear  visualization  of  depth. 

The  software  application  calibrates  the  cameras  using  the  plane  based  calibration  model  from  the  OpenCV 
calib3D  module  and  allows  the  cameras  to  be  calibrated  before  being  mounted  in  the  tire.  The  calibration 
process  consists  of  the  following  steps: 

1 .  Multiple  snapshots  are  taken  of  a  calibration  surface  with  both  cameras. 
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2.  An  optimization  is  performed  to  calculate  the  distortion  coefficients  which  will  allow  the  lens  distortion 
to  be  removed  from  each  image  individually. 

3.  The  rotational  and  translational  matrices  between  the  two  cameras  are  calculated  from  the  calibration 
plane  data  and  the  two  camera  views  are  re-projected  to  form  horizontal  epipolar  lines. 


(c)  Remove  vertical  disparity  to  create  epipolar  lines 
Figure  2  -  Calibrating  the  cameras  and  removing  disparities  in  images 


The  epipole  is  the  re-projection  of  the  cameras  center  of  projection  onto  the  other  cameras  image  plane. 
Thus  the  epipole  is  the  image,  in  one  camera,  of  the  optical  center  of  the  other  camera.  An  epipolar  line  is  the 
projection  of  a  ray,  which  passes  through  the  center  of  projection,  from  the  other  camera.  All  epipolar  lines 
intersect  at  the  epipole.  The  process  of  aligning  the  images  along  epipolar  lines  is  called  rectification.  The 
process  re-projects  the  two  camera  views  such  that  an  object  projected  onto  both  cameras  lie  on  the  same 
horizontal  line.  Corresponding  pixels  between  the  two  views  can  be  obtained  by  simple  line  searches.  This 
process  is  vital  for  fast  and  accurate  calculation  of  the  horizontal  disparity. 

Once  the  epipolar  lines  are  determined  the  software  is  able  to  search  along  these  lines  for  corresponding 
pixel  areas  (shown  in  Figure  3)  and  compute  the  horizontal  disparity  using  either  a  Block  Matching  or  Semi- 
Global  Block  Matching  algorithm.  The  disparity  is  then  used  to  calculate  a  3-D  point  cloud  which  creates  a 
visual  of  what  is  happening  inside  the  tire  as  shown  in  Figure  3. 

A  screenshot  of  the  stereovision  software  is  shown  in  Figure  4.  The  figure  shows  the  user  interface  with 
rectified  images  from  both  the  left  and  right  videos  captured  from  inside  the  tire.  A  colored  depth  map  is  also 
shown.  The  map  is  colored  according  to  the  distance  from  the  camera,  with  red  signifying  points  is  closer  to 
the  camera  and  blue  further  away.  The  curve  of  the  tire  and  sidewall  is  observable  even  in  this  simple  colored 
map.  The  white  areas  around  the  edges  of  the  image  indicate  features  are  not  visible  in  both  camera  images. 
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(a)  Search  along  epipolar  lines  for  corresponding  pixel  areas  and 


compute  disparity 


(b)  Calculate  3D  point  cloud  from  disparity 


Figure  3  -  Creating  a  3-D  point  cloud  from  disparity 
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Figure  4  -  Screenshot  of  the  stereovision  software  interface  with  an  exampie  of  a  rectified  stereo  image  pair 
and  the  corresponding  disparity  map  (colored  with  red  being  the  closest  points  to  the  camera  and  blue  being 
the  furthest. 
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6.  Hardware 

Figure  5  indicates  the  original  design  of  the  hardware  to  be  fitted  inside  the  tire  (left),  compared  to  the  actual 
hardware  manufactured  (right).  The  rim  was  painted  white  to  reflect  the  light  from  the  LED’s  evenly  towards 
the  tire. 


Stationary 
Ring  Mount 


Outer  Stationary 
Rtng 


VDG  Wteel  Force 

Transducer 


Outer  Pinhon 
Pulley 


innar  Pinion 

Pulley 


Figure  5  -  Description  of  complete  T2-CAM  system  with  VDG  wheel  force  transducer 


The  prototype  T2-CAM  system  was  assembled  inside  the  tire.  The  complete  T2-CAM  system,  in  conjunction 
with  the  VDG  wheel  force  transducer,  is  mounted  on  a  test  vehicle  as  indicated  in  Figure  6.  Several  static 
tests  were  performed  including  vertical  loading  on  flat  surface  as  well  as  cleats  at  different  angles.  Dynamic 
tests  were  performed  with  the  vehicle  driving  lateral  and  longitudinal  cleats  as  well  as  obstacles  of  different 
sizes. 


Figure  6  -  Complete  T2-CAM  system  with  VDG  wheel  force  transducer  fitted  to  test  vehicle 
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7.  Results 

The  most  exciting  and  intuitive  test  results  were  obtained  by  driving  over  cleats.  Figure  97  shows  the  3-D 
point  cloud  obtained  from  the  stereo  vision  system  (colored  points)  superimposed  on  the  camera  image  (dark 
grey  with  white  speckle  pattern).  The  flat  contact  area  when  the  tire  is  moving  on  the  level  road  can  be  seen. 
The  color  is  scaled  according  to  the  vertical  displacement  of  the  tire  e.g.  blue  is  low  and  red  is  high.  The 
image  disappears  beyond  the  red  part  of  the  spectrum  because  these  parts  of  the  side  wall  of  the  tire  are  not 
visible  in  both  camera  images  and  correspondences  can  therefore  not  be  found. 

Figure  8  (left)  shows  the  corresponding  results  when  the  vehicle  is  driven  over  a  longitudinal  cleat  (round  bar) 
as  indicated  in  Figure  8  (right).  The  three-dimensional  deformation  of  the  tire  can  be  clearly  seen. 

Making  lateral  cuts  through  the  three-dimensional  image  result  in  lines  representing  the  vertical  deformation 
of  the  tire  at  specified  longitudinal  positions  as  indicated  in  Figure  9.  It  should  be  noted  that  these 
displacements  are  referenced  relative  to  the  camera  and  not  to  the  ground. 


Figure  7  -  Three-dimensional  deformation  with  tire  on  level  surface 


Figure  8  -  Three-dimensional  surface  for  tire  driving  over  longitudinal  cleat  (left)  as  well  as  test  setup  on 
vehicle  (right) 
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Figure  9  -  Lateral  cuts  from  3-D  image  at  different  longitudinal  positions  when  driving  over  a  longitudinal 
cleat 
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10.  Administrative  actions 

None 

11.  Other  important  information 

The  project  has  been  successfully  completed.  The  project  team  also  gave  formal  feedback  to  representatives 
from  ERDC-GSL,  ERDC-CRREL  and  ERDC-IRO  on  3  August  2015  at  ERDC-CRREL  in  Hanover,  NH. 


